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MEMORANDUM  FOR  UNDER  SECRETARY  OF  DEFENSE  FOR  ACQUISITION, 

TECHNOLOGY,  AND  LOGISTICS 

SUBJECT:  Improvements  Needed  to  the  Purchase  Card  On-Line  System  (Report  No.  DODIG- 


2013-061) 


We  are  providing  this  report  for  your  information  and  use.  No  written  response  to  this  report 
was  required,  and  none  was  received.  However,  informal  comments  from  management  on  a 
discussion  draft  were  considered  in  preparing  the  report.  We  are  publishing  this  report  in  final 
form.  The  Director  of  Program  Development  and  Implementation,  Defense  Procurement  and 
Acquisition  Policy,  Office  of  the  Under  Secretary  of  Defense  (Acquisition,  Technology,  & 
Logistics),  requested  this  audit.  Our  objective  was  to  determine  whether  Department  of  Defense 
Approving/Billing  Officials  adequately  reviewed  transactions  that  the  Purchase  Card  On-Line 
System  referred  for  being  at-risk  of  noncompliance  with  applicable  laws  and  criteria. 

Neither  we  nor  the  Defense  Procurement  and  Acquisition  Policy  could  use  the  Purchase  Card 
On-Line  System  to  assess  and  determine  whether  DoD  Approving/Billing  Officials  adequately 
reviewed  32,690  transactions  that  the  system  referred  (during  the  period  January  through 
June  2012)  as  being  at-risk  of  noncompliance  with  applicable  laws  and  criteria,  including  lost 
and  stolen  cards. 

When  we  requested  information  to  conduct  the  audit,  the  Defense  Procurement  and  Acquisition 
Policy  office  became  aware  the  automated  system  lacked  the  capability  to  produce  the  data 
needed  to  complete  the  assessment  of  the  Approving/Billing  Officials’  reviews  and  initiated 
actions  to  improve  the  Purchase  Card  On-Line  System.  Thus,  we  are  not  making  any 
recommendations  in  this  report. 


We  appreciate  the  courtesies  extended  to  the  staff.  Please  direct  questions  to  me  at 
(703)  601-5945. 


Lorin  T.  Venable,  CPA 

Acting  Assistant  Inspector  General 

DoD  Payments  and  Accounting  Operations 
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Results  in  Brief:  Improvements  Needed  to 
the  Purchase  Card  On-Line  System 


What  We  Did 

The  Director  of  Program  Development  and 
Implementation,  Defense  Procurement  and 
Acquisition  Policy,  Office  of  the  Under 
Secretary  of  Defense  (Acquisition,  Technology, 
&  Logistics),  requested  this  audit.  Our 
objective  was  to  detennine  whether  Department 
of  Defense  Approving/Billing  Officials 
adequately  reviewed  transactions  that  the 
Purchase  Card  On-Line  System  referred  for 
being  at-risk  of  noncompliance  with  applicable 
laws  and  criteria.  In  addition,  we  were  to 
determine  whether  Department  of  Defense 
Approving/Billing  Officials  adequately 
reviewed  cases  in  which  they  identified  cards  as 
lost  or  stolen. 

What  We  Found 

Neither  we  nor  the  Defense  Procurement  and 
Acquisition  Policy  could  use  the  Purchase  Card 
On-Line  System  to  assess  and  determine 
whether  DoD  Approving/Billing  Officials 
adequately  reviewed  32,690  transactions  that  the 
system  referred  (during  the  period  January 
through  June  2012)  as  being  at-risk  of 
noncompliance  with  applicable  laws  and 
criteria,  including  lost  and  stolen  cards. 


As  a  result,  the  automated  system  cannot  be 
employed  for  oversight  reviews,  such  as 
assessments  of  Approving/Billing  reviews  and 
disposition  of  at-risk  transactions. 

Because  of  the  information  we  requested  to 
conduct  the  audit,  the  Defense  Procurement  and 
Acquisition  Policy  became  aware  the  automated 
system  lacked  the  capability  to  produce  the  data 
needed  to  complete  the  assessment  of  the 
Approving/Billing  Officials’  reviews  and 
initiated  actions  to  improve  the  Purchase  Card 
On-Line  System.  Thus,  we  are  not  making  any 
recommendations  in  this  report. 

Management  Comments  and 
Our  Response 

We  do  not  require  a  written  response  to  this 
report. 


This  occurred  because  the  automated  system: 


•  did  not  contain  sufficient  capability  to 
automatically  retrieve  and  match  the 
case  disposition  reviews  with  the 
universe  of  the  at-risk  Government 
Purchase  Card  transactions,  and 

•  was  unable  to  archive  case  history  file 
data,  which  were  maintained  in  a 
separate  data  warehouse. 
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Introduction 


Objectives 

The  Director  of  Program  Development  and  Implementation,  Defense  Procurement  and 
Acquisition  Policy  (DPAP),  Office  of  the  Under  Secretary  of  Defense  (Acquisition, 
Technology  &  Logistics),  requested  this  audit.  Our  objective  was  to  determine  whether 
DoD  Approving/Billing  Officials  (A/BOs)  adequately  reviewed  transactions  that  the 
Purchase  Card  On-Line  System  (PCOLS)  referred  for  being  at-risk  of  noncompliance 
with  applicable  laws  and  criteria.  In  addition,  we  were  to  determine  whether  DoD  A/BOs 
adequately  reviewed  cases  in  which  they  identified  cards  as  lost  or  stolen.  See  the 
Appendix  for  the  scope  and  methodology  related  to  the  objective. 

Government  Purchase  Card  Audit  Requirements 

Section  2784,  title  10,  United  States  Code  (10  U.S.C.  §  2784)  establishes  the  Government 
Purchase  Card  (GPC)  Program  for  the  Department  of  Defense.  Section  2784  requires  the 
DoD  Office  of  Inspector  General  to  perform  periodic  audits  of  the  DoD  GPC  Program  to 
identify  the  following: 

•  potentially  fraudulent,  improper,  and  abusive  use  of  purchase  cards; 

•  patterns  of  improper  cardholder  transactions,  such  as  purchases  of  prohibited 
items;  and 

•  categories  of  purchases  that  should  be  made  by  means  other  than  the  GPC  to 
better  aggregate  purchases  and  obtain  lower  prices. 

Furthermore,  section  2784  requires  DoD  to  use  “effective  systems,  techniques,  and 
technologies  to  prevent  or  identify  potential  fraudulent  purchases.” 

DoD  Office  Responsible  for  the  Government  Purchase 
Card  Program 

The  DPAP  was  responsible  for  oversight  and  issuing  policy  for  the  DoD  GPC  Program. 
To  assist  DoD  with  internal  control  and  policy  compliance,  the  DPAP  awarded  a  contract 
for  the  development  of  a  Data  Mining  and  Risk  Assessment  (DMRA)  applications  as  part 
of  an  automated  system,  the  PCOLS,  to  examine  transactions  and  identify  those  that  were 
at-risk  of  being  noncompliant  with  laws  and  other  criteria.  DoD  began  deployment  of  the 
PCOLS  Data  Mining  and  Risk  Assessment  modules  during  the  2nd  Quarter  of  FY  2009  at 
one  Air  Force  location,  the  Defense  Contract  Management  Agency,  the  Defense  Logistics 
Agency,  and  the  Washington  Headquarters  Services  and  that  deployment  continues.  As 
of  February  6,  2013,  the  PCOLS  Data  Mining  and  Risk  Assessment  module  contract  ceiling 
amount  was  $14.4  million. 
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The  Purchase  Card  On-Line  System 

The  DPAP  developed  the  PCOLS  to  improve  controls  over  the  use  of  the  GPC  program 
by  using  a  data  mining  process  to  identify  potentially  improper  GPC  transactions.  The 
PCOLS  suite  of  applications  consists  of  the  following  modules: 

•  Enterprise  Monitoring  and  Management  of  Accounts, 

•  Authorization,  Issuance,  and  Maintenance, 

•  Data  Mining, 

•  Risk  Assessment,  and 

•  PCOLS  Reporting. 

The  Enterprise  Monitoring  and  Management  of  Accounts  module  and  the  Authorization, 
Issuance,  and  Maintenance  module  were  administrative  modules.  The  Data  Mining 
module  contained  a  mining  feature  that  was  capable  of  reviewing  100  percent  of  the 
purchase  card  transactions  to  provide  near  real-time  transaction  monitoring.  As  part  of 
the  function  of  the  DMRA  modules,  each  transaction  was  assigned  a  Risk  Predictive 
Model  score  that  identified  the  risk  of  impropriety  for  each  transaction.  Selected 
transactions  with  a  high  Risk  Predictive  Model  score  and  randomly  selected  transactions 
were  flagged  for  management  review. 

Under  the  DMRA  operations  concept,  the  primary  or  the  alternate  A/BOs,  who  received  a 
notification  e-mail,  were  to  review  each  of  the  flagged  transactions.  The  A/BOs  were  to 
determine  the  validity  of  the  transaction.  With  every  flagged  transaction,  the  A/BOs 
were  to  answer  a  minimum  of  nine  questions,  such  as  whether  the  transaction  was  a  split 
transaction  or  a  prohibited  item.  This  review  process  was  termed  “case  review.”  After 
the  case  review,  the  A/BO  would  complete  a  “case  disposition”  to  categorize  the 
transaction  as  valid,  administrative  discrepancy,  misuse,  suspected  fraud,  lost,  or  stolen. 
Also,  the  GPC  transactions  that  the  DMRA  identified  as  “high  risk”  transactions  were  to 
undergo  an  independent  review  by  the  responsible  Agency/Organization  Program 
Coordinator.  Recently,  DPAP  decided  to  migrate  the  Risk  Assessment  functionality  to 
the  Reporting  Module. 

Review  of  Internal  Controls 

DoD  Instruction  5010.40,  “Managers’  Internal  Control  Program  (MICP)  Procedures,” 

July  29,  2010,  requires  DoD  organizations  to  establish  a  management  internal  control 
program  to  identify  and  promptly  correct  ineffective  internal  controls  and  establish 
internal  controls  when  warranted.  We  identified  an  internal  control  weakness:  PCOLS 
did  not  contain  a  sufficient  capability  to  automatically  retrieve  and  match  the  case 
dispositions’  reviews  with  the  data  on  the  at-risk  GPC  transactions.  During  the  audit,  the 
DPAP  was  taking  actions  to  correct  the  weaknesses.  We  will  provide  a  copy  of  the  report 
to  the  senior  official  responsible  for  internal  controls  in  the  DPAP. 
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Finding.  Adequacy  of  Data  Availability 

Neither  we  nor  the  Defense  Procurement  and  Acquisition  Policy  could  use  the  PCOLS 
System  to  assess  and  determine  whether  DoD  A/BOs  adequately  reviewed  32,690 
transactions  that  the  system  referred  (during  the  period  January  through  June  2012)  as 
being  at-risk  of  noncompliance  with  applicable  laws  and  criteria,  including  lost  and 
stolen  cards. 

This  occurred  because  the  PCOLS: 

•  did  not  contain  sufficient  capability  to  automatically  retrieve  and  match  the  case 
disposition  reviews  with  the  universe  of  the  at-risk  GPC  transactions,  and 

•  was  unable  to  archive  case  history  file  data,  which  were  maintained  in  a  separate 
data  warehouse. 

As  a  result,  the  automated  system  could  not  be  employed  for  oversight  reviews,  such  as 
assessments  of  A/BOs  reviews  and  dispositions  of  at-risk  transactions. 

Audit  Request 

On  August  24,  2012,  the  Director  of  Program  Development  and  Implementation,  DPAP, 
Office  of  the  Under  Secretary  of  Defense  (Acquisition,  Technology  &  Logistics), 
requested  this  audit,  and  we  announced  the  audit  on  September  5,  2012. 

Complete  Universe  Data  Availability  Was  Limited 

Although  the  PCOLS  provided  the  Military  Departments  and  Defense  Agencies  with 
another  oversight  capability  not  previously  available  to  everyone,  the  PCOLS  capability 

to  allow  overall  assessments  was  limited.  The 
PCOLS  provided  a  data  mining  tool  that  identified 
transactions  that  were  potentially  noncompliant 
with  applicable  laws  and  criteria.  However,  since 
the  PCOLS  was  deployed  in  2009,  it  did  not  have 
the  ability  to  provide  the  information  that  was 
needed  to  resolve  whether  approving  officials 
adequately  reviewed  the  transactions  that  the  PCOLS  referred  as  being  at-risk  of 
noncompliance,  which  was  also  the  primary  objective  of  this  management-requested 
audit. 

On  October  1,  2012,  the  DPAP  DMRA  application  provided  148,488  cases  for 
management  review  during  the  period  of  January  1,  2012,  through  June  30,  2012.  The 
148,488  universe  included  32,690  cases  that  A/BOs  reviewed  and  closed.  However,  the 
remaining  1 15,798  cases  were  either  still  in  the  review  process  or  were  related  to  Military 
Departments  and  DoD  Agencies  not  using  PCOLS.  For  example,  the  Army  and  Navy 
were  not  fully  using  PCOLS  for  the  review  of  at-risk  transactions;  however,  their 
transactions  were  included  in  the  data  received  from  the  banks  that  provide  DoD  GPCs. 


The  PCOLS  provided  a  data 
mining  tool  that  identified 
transactions  that  were 
potentially  noncompliant  with 
applicable  laws  and  criteria 


3 


When  we  requested  information  to  conduct  the  audit,  the  DPAP  personnel  became  aware 

the  DMRA  database  did  not  contain  sufficient 
capability  to  automatically  retrieve  and  match  the 
case  disposition  reviews  with  the  universe  of  the 
at-risk  GPC  transactions.  For  management  to 
perform  an  assessment  of  the  A/BOs’  review 
results  after  case  disposition  closure,  the  DMRA 
needed  to  match  two  sets  of  data.  The  DMRA 
needed  to  match  the  following: 

•  transactions  referred  to  the  A/BOs  for  review  to  the  responses  that  the  A/BOs 
provided  regarding  the  case,  and 

•  specific  information  that  identified  the  transactions’  cardholders,  and  other  data, 
such  as  locations,  vendors’  data,  and  time  frames  for  the  review  process. 

Furthermore,  as  a  result  of  the  inability  to  automatically  match  the  two  sets  of  data,  the 
dollar  amount  and  other  summary  data  that  the  A/BOs’  cases  represented  were  not  able  to 
be  retrieved  in  an  efficient  and  timely  manner.  On  October  22,  2012,  DPAP  personnel 
admitted  that  the  system  could  not  automatically  produce  the  information  requested  and 
that  obtaining  this  information  would  require  a  significant  manual  effort. 

Historical  POOLS  Data  Were  Not  Being  Archived  in  the 
Data  Warehouse 

The  DPAP  was  not  aware  that  POOLS  Data  were  not  being  archived  in  a  data  warehouse 

before  our  request  for  information  to  support  the  audit 
objective.  DPAP  personnel  stated,  “We  were  under  the 
impression  that  the  requested  data  would  be  available 
from  the  POOLS  Reporting  Data  Warehouse. 
Unfortunately,  what  we  found  out  is  that  it  is  not.” 
DPAP  personnel  also  stated,  “DPAP  was  unaware  of  deficiencies  in  the  data  warehouse 
when  the  audit  commenced.  The  data  existed  but  was  [sic]  not  readily  available.”  On 
January  8,  2013,  DPAP  personnel  through  e-mail  explained  the  challenges  of  obtaining 
POOLS  data  to  support  the  audit: 

When  DPAP  requested  the  audit  in  August  of  2012,  the  expectations  were  that  PCPO1 
would  be  able  to  provide  the  data  required  for  the  audit  through  ad  hoc  queries  run 
against  the  PCOLS  data  warehouse.  PCPO  tasked  DMDC  [Defense  Manpower  Data 
Center]  with  performing  these  ad  hoc  queries  and  the  data  made  available.  DMDC 
performed  the  queries  and  supplied  the  extracted  data  to  PCPO,  which,  was  [sic] 
incomplete  for  the  scope  of  this  audit.  It  is  important  to  note  that  all  of  the  data  from 
[Data  Mining]  activity  is  [sic]  transferred  to  the  DMDC.  Heretofore,  only  the  data 
required  for  PCOLS  Reporting  purposes  has  [sic]  been  loaded  into  the  PCOLS  data 
warehouse.  It  was  not  until  after  close  examination  of  the  extracted  data  did  PCPO 


PCOLS  Data  were  not 
being  archived  in  a 
data  warehouse 


DMRA  database  did  not  contain 
sufficient  capability  to 
automatically  retrieve  and 
match  the  case  disposition 
reviews  with  the  universe  of  the 
at-risk  GPC  transactions 


1  The  Purchase  Card  Policy  Office  (PCPO)  is  part  of  the  Program  Development  and  Implementation 
Directorate,  Defense  Procurement  and  Acquisition  Policy,  Office  of  the  Under  Secretary  of  Defense 
(Acquisition,  Technology  &  Logistics)  that  requested  the  audit. 
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become  aware  that  all  of  the  historical  data  was  [sic]  not  being  stored  in  the  data 
warehouse.  The  elements  of  this  historical  data  were  the  cause  of  the  audit  data  being 
incomplete. 

In  addition,  DPAP  personnel  stated: 

We  determined  that  some  of  the  data  fields  that  needed  to  be  queried  were  not 
permanently  stored  in  the  data  warehouse.  These  required  fields  are  summarily  extracted 
to  worktables,  calculations  made  to  summarize  the  data,  and  then  the  work  tables  deleted. 
The  result  was  that  the  some  [sic]  of  the  historical  data  was  [sic]  not  retained  in  the  data 
warehouse  and  subsequently  not  available  for  immediate  retrieval. 

Defense  Procurement  and  Acquisition  Policy  Improving 
the  Purchase  Card  On-Line  System  To  Permit  Added 
Oversight 

DPAP  personnel  indicated  they  were  making  improvements  that  will  result  in  retention  of 
the  data  necessary  to  assess  A/BOs  performance  and  also  permit  the  PCOLS  to  be  used  to 
identify  trends,  develop  metrics,  and  focus  on  problem  areas.  Specifically,  on 
January  8,  2013,  DPAP  personnel  e-mailed  the  following: 

In  order  to  ensure  the  ability  to  re-engage  with  your  Office  to  complete  this  audit  when  it 
is  appropriate,  and  ensure  efficient  use  of  audit  resources,  DPAP  has  taken  actions  to 
ensure  a  higher  degree  of  audit  preparedness  with  respect  to  data  accessibility  and  system 
architecture  in  the  Data  Mining  application  suite.. . .  We  expect  that  this  data  to  be  loaded 
into  production  by  the  end  of  FY  2013  Q2  and  available  for  adhoc  [sic]  reporting.  We 
expect  that  audit  specific  reports  to  be  available  in  production  by  the  end  of  FY  2013  Q3. 

Other  Events  Affecting  Data  Availability 

The  DPAP  personnel  subsequently  indicated  that  they  were  making  changes  to  the 
PCOLS  that  would  further  affect  the  availability  of  data.  Specifically,  the  DPAP 
completed  moving  the  host  for  the  PCOLS  Data  Mining  module  from  a  contractor’s  site 
to  the  Defense  Systems  Information  Agency  Enclave  (a  DoD  environment)  on 
January  26,  2013.  The  DPAP  expected  the  move  would  add  to  the  delay  in  obtaining 
data. 

The  DPAP  personnel  expected  that  the  move  to  the  Defense  Systems  Information  Agency 

Enclave  would  have  a  positive  effect  on 
future  programming  and  maintenance  of  the 
Data  Mining  application.  Additionally,  the 
DPAP  expected  the  move  would  instill 
tighter  controls  related  to  software 
configuration  management  and  overall  database  support  once  the  application  was  within 
the  Defense  Systems  Information  Agency  environment.  Both  these  factors  were  expected 
to  improve  software  quality  and  overall  application  quality. 


DPAP  personnel  expected  that  the 
move  to  the  Defense  Systems 
Information  Agency  Enclave  would 
have  a  positive  effect 
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Summary 

Until  DPAP  personnel  effectively  complete  their  efforts  to  develop  the  methodology  that 
will  allow  a  complete  matching  of  case  disposition  information  with  specific  GPC 
transactions,  they  will  not  have  the  ability  to  perfonn  an  effective  assessment  of  the 
A/BOs  review  process.  Additionally,  any  audit  reviews  will  be  limited  and  the  PCOLS 
cannot  be  used  to  the  full  potential  for  managing  the  GPC  program. 

DPAP  personnel  recognized  the  deficiencies  that  they  had  in  the  automated  system  that 
were  preventing  them  from  obtaining  the  data  needed  for  effective  oversight  and  initiated 
the  actions  mentioned  above  to  improve  the  PCOLS.  As  a  result  of  management  actions 
to  be  taken  in  the  2nd  and  3rd  Quarters  of  FY  2013,  we  are  not  making  recommendations 
in  this  report. 
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Appendix:  Scope  and  Methodology 

We  conducted  this  performance  audit  from  September  2012  through  March  2013  in 
accordance  with  Generally  Accepted  Government  Auditing  Standards.  However,  the 
DPAP  was  unable  to  provide  sufficient  information  to  allow  for  selecting  of  an  audit 
sample  to  review  the  case  disposition  process.  The  standards  require  that  we  plan  and 
perform  the  audit  to  obtain  sufficient,  appropriate  evidence  to  provide  a  reasonable  basis 
for  our  findings  and  conclusions  based  on  our  audit  objectives.  We  believe  that  the 
evidence  obtained  provides  a  reasonable  basis  for  the  findings  and  conclusions  we 
formed  based  on  our  audit  objectives. 

To  determine  whether  A/BOs  adequately  reviewed  transactions  that  the  PCOLS  referred 
for  being  at-risk  of  noncompliance  with  applicable  laws  and  criteria,  we  interviewed 
DPAP  personnel  concerning  the  PCOLS  case  disposition  process  and  reviewed 
applicable  criteria.  Additionally,  we  requested  the  DPAP  to  provide  the  January  1,  2012, 
through  June  30,  2012,  universe  of  at-risk  transactions  referred  by  the  PCOLS  to  the 
A/BOs  to  select  a  sample  to  review.  On  October  1,  2012,  DPAP  provided  148,488  cases 
created  by  the  PCOLS  to  be  sent  to  A/BOs  for  review  during  the  period  of  January  1 , 
2012,  through  June  30,  2012.  The  148,488  cases  included  32,690  cases  that  the  A/BOs 
had  reviewed  and  closed.  However,  the  remaining  1 15,798  cases  were  either  still  in  the 
review  process  or  were  related  to  Military  Departments  and  DoD  Agencies  not  utilizing 
PCOLS.  For  example,  the  Army  and  Navy  were  not  fully  utilizing  PCOLS  for  the  review 
of  at-risk  transactions;  however,  their  transactions  were  included  in  the  data  PCOLS 
received  from  the  banks  and  at-risk  cases  were  created  in  PCOLS. 

However,  because  of  the  PCOLS ’s  system  limitations,  the  DPAP  was  unable  to  provide 
sufficient  information  for  us  to  review  the  32,690  cases.  The  PCOLS  did  not  have  the 
ability  to  automatically  match  the  necessary  data  to  conduct  the  audit  including  A/BOs’ 
case  disposition  results  matched  to  information  that  identified  the  specific  transactions, 
card  holders,  locations,  vendors’  data,  and  time  frames  for  the  review  process. 

Use  of  Computer-Processed  Data 

We  obtained  computer-processed  data  from  the  DPAP  in  support  of  the  audit  objective. 
However,  the  PCOLS  data  did  not  contain  the  necessary  information  to  conduct  the  audit. 
The  PCOLS  did  not  contain  A/BOs’  case  disposition  results  matched  to  information  that 
identified  the  specific  transactions,  card  holders,  locations,  vendors’  data,  and  time 
frames  for  the  review  process.  As  a  result,  computer-processed  data  were  not  used  to 
conduct  this  audit. 

Prior  Coverage 

No  prior  coverage  has  been  conducted  on  the  use  of  the  Purchase  Card  On-Line  System 
to  assess  and  detennine  whether  DoD  A/BOs  adequately  reviewed  transactions  referred 
to  them  for  being  at-risk  of  noncompliance  with  applicable  laws  and  criteria  during  the 
last  5  years. 
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